Benchmarking vision kernels and neural network inference accelerators on embedded platforms

Developing efficient embedded vision applications requires exploring various algorithmic optimization trade-offs and a broad spectrum of hardware architecture choices. This makes navigating the solution space finding design points with optimal performance challenge for developers. To help provide fair baseline comparison, we conducted comprehensive benchmarks accuracy, run-time, energy efficiency wide range kernels neural networks on multiple platforms: ARM57 CPU, Nvidia Jetson TX2 GPU Xilinx ZCU102 FPGA. Each platform utilizes their optimized libraries (OpenCV, VisionWorks xfOpenCV) (OpenCV DNN, TensorRT DPU). For kernels, our results show that achieves an energy/frame reduction ratio 1.1–3.2× compared to others simple kernels. However, more complicated complete pipelines, FPGA outperforms ratios 1.2–22.3×. [Inception-v2 ResNet-50, ResNet-18, Mobilenet-v2 SqueezeNet], it shows speed up [2.5, 2.1, 2.6, 2.9 2.5]× EDP [1.5, 1.1, 1.4, 2.4 1.7]× FP16 implementations, respectively.